{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "employed-cloud",
   "metadata": {},
   "source": [
    "# Morris Sensitivity Analysis\n",
    "\n",
    "Link to API Reference: [MorrisSensitivity](./python/api/MorrisSensitivity.ipynb)\n",
    "\n",
    "*See the backing repository for Morris [here](https://github.com/SALib/SALib).*\n",
    "\n",
    "<h2>Summary</h2>\n",
    "\n",
    "Also known as the Morris method[[1](morris1991factorial_msa)], this is a one-step-at-a-time (OAT) global sensitivity analysis where only one input has its level (discretized value) adjusted per run. Relative to other sensitivity analysis algorithms, the Morris method is fast (fewer model executions) but comes at the cost of not being able to differentiate non-linearities with interactions. This is commonly used for screening which inputs are important enough for further analysis. The implementation uses `SALib`[[2](herman2017salib_msa)] for its Morris method.\n",
    "\n",
    "<h2>How it Works</h2>\n",
    "\n",
    "The `gsa-module` package has a good conceptual overview of the Morris method for screening [here](https://gsa-module.readthedocs.io/en/stable/implementation/morris_screening_method.html).\n",
    "\n",
    "The `SALib` package describes what sensitivity analysis is, and the steps required in conducting it at a basic level [here](https://salib.readthedocs.io/en/latest/basics.html).\n",
    "\n",
    "The conceiving paper for Morris method[[1](morris1991factorial_msa)] can be found [here](https://abe.ufl.edu/Faculty/jjones/ABE_5646/2010/Morris.1991%20SA%20paper.pdf).\n",
    "\n",
    "<h2>Code Example</h2>\n",
    "\n",
    "The following code will train a blackbox pipeline for the breast cancer dataset. Aftewards it will interpret the pipeline and its decisions with Morris method. The visualizations provided will be for global explanations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "arbitrary-schedule",
   "metadata": {},
   "outputs": [],
   "source": [
    "from interpret import set_visualize_provider\n",
    "from interpret.provider import InlineProvider\n",
    "set_visualize_provider(InlineProvider())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "moved-fabric",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.datasets import load_breast_cancer\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.decomposition import PCA\n",
    "from sklearn.pipeline import Pipeline\n",
    "\n",
    "from interpret import show\n",
    "from interpret.blackbox import MorrisSensitivity\n",
    "\n",
    "seed = 42\n",
    "np.random.seed(seed)\n",
    "X, y = load_breast_cancer(return_X_y=True, as_frame=True)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=seed)\n",
    "\n",
    "pca = PCA()\n",
    "rf = RandomForestClassifier(random_state=seed)\n",
    "\n",
    "blackbox_model = Pipeline([('pca', pca), ('rf', rf)])\n",
    "blackbox_model.fit(X_train, y_train)\n",
    "\n",
    "msa = MorrisSensitivity(blackbox_model, X_train)\n",
    "\n",
    "show(msa.explain_global())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fourth-armstrong",
   "metadata": {},
   "source": [
    "<h2>Further Resources</h2>\n",
    "\n",
    "- [Wikipedia on Morris Method](https://en.wikipedia.org/wiki/Morris_method)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "black-arcade",
   "metadata": {},
   "source": [
    "<h2>Bibliography</h2>\n",
    "\n",
    "(morris1991factorial_msa)=\n",
    "[1] Max D Morris. Factorial sampling plans for preliminary computational experiments. Technometrics, 33(2):161–174, 1991.\n",
    "\n",
    "(herman2017salib_msa)=\n",
    "[2] Jon Herman and Will Usher. Salib: an open-source python library for sensitivity analysis. Journal of Open Source Software, 2(9):97, 2017."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
